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ABSTRACT 

The use of indicators as tools lor educational 
improvement is discussed. National indicators are limited in breadth 
and depth and are neither sensitive to differences across states nor 
detailed and specific enough to be useful to states. For indicators 
to contribute to the goals of local educational improvement, local 
agencies must define their own indicators, and district policymakers 
must pass on similar discretion to their schools. For a system of 
indicators to be useful for educational improvement, it must provide 
adequate measures of those aspects of schooling deemed important and 
amenable to change through policymaking. A useful system of 
indicators requires measures of the content and qualitj^ of 
instruction and analyses and presentation of data directed to 
specific policy issues. The following organizational factors help 
policymakers use data to make decisions: (1) a climate that supports 
planning and use of data; (2) commitment to improvement by disv.ict 
leaders; (3) stakeholder involvement in designing the data system; 
(4) technical expertise and data system support; and (5) an action 
system and resource for change. Indicators should match curricular 
goals and reflect the content and quality of instruction. Analyses of 
existing data should provide insight beyond aggregate measures; and 
school staff should be involved in specification, collection, 
analysis, and interpretation of data. (TJH) 
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SUMMARY 



Indicators are usually discussed in the context of accountability. 
In this context, it is assumed that rewards tied to high performance 
on indicators of effectiveness will stimulate low-performing schools to 
improve. Thus, indicators are viewed as incentives to change via 
external pressure. The author relieves that significant change is 
more likely to result from a system designed to foster professional 
responsibility for self improvement than one that relies on external 
control. Therefore, this paper focuses on the usefulness of 
indicators as guides for educational improvement. 

The use of indicators as tools for self improvement does not 
preclude their being used for accountability purposes, however. If 
districts create systems of indicators that are useful to policymakers 
in improving education quality, they are necessarily useful for 
purposes of accountability to the public. 

V 

National indicators, by definition, are limited in breadth and 
depth, and are neither sensitive to differences across states nor 
detailed and specific enough to be useful to states. For indicators 
to contribute to goals of local improvement, local agencies must define 
their own indicators, and district policymakers should in turn pass on 
similar discretion to their schools. Therefore, this paper does not 
consider a multi- tiered system of indicators created above the level of 
local education agercxes. 

Most districts currently collect a lot of operational data. Yet, 
these data are rarely usea to inform policymaking. In order to 
promote the use of data as iadicators for educational improvement, 
districts must first ensure that they are collecting the right kinds of 
data and that conditions exist for optimum use of these data. 

For a system of indicators to be useful for educational 
improvement is it must provide adequate measures of those aspects of 
schooling deemed important and amenable to change through policymaking. 
A useful system of indicators therefore requires: 

-measures of the content and quality of instruction; and 
-analyses and presentation of data directed to specific 
policy issues. 

In addition, five organizational factors help policymakers use data to 
make decisions: 

1. a climate that supports planning and use of data; 

2. comndtment to improvement by district leaders; 

3. stakeholder involvement in designing the data system; 

4. technical expertise and data system support, and 

5. an action system and resources for change. 



For indicators to motivate district and school stafJ to change the 
way they do things, their selection should be accompanied by attention 
to the organizational factors that promote feedback for improvement at 
all levels of the system. Data alone have never been a catalyst for 
change. This suggests the need for cooperative planning among 
policymakers, analysts, and school staffs to create a workable system. 
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I . INTRODUCTICm 



"Education indicators'* are defined as single or composite 
statistics reflecting the health of the educational system that can be 
readily, reliably, and repeatedly obtained, thus permitting comparisons 
over time and among states or districts. (Johnstone 1981; Kaagan and 
Smith, 1985; National Center for Education Statistics, 1985). In this 
paper, I consider ways in which indicators can help local policymakers 
improve the quality of their schools. 

Indicators recently have received considerable attention as a way 
to meet the needs of federal and state policymakers for comparative 
datu on education quality. The difficulties of creating a system of 
indicators tchat adequately reflects education performance have been 
well -documented, including the need for common definitions, measures 
that match and adequately reflect education goals, and methods for 
insuring fairness of comparisons across very different subunits. 
(Baker 1985; National Center for Education Statistics, 1985; Oakes, 
1986). 

Indicators are usually discussed in the context of accountability. 
In this context, it is assumed that rewards tied to high performance on 
indicators of effectiveness will stimulate low performing schools to 
improve. Thus, indicators are viewed as incentives to change via 
external pressure. 

I choose instead to focus on the usefulness of indicators as guides 
for education improvement. This choice derives from my belief that 
significant change is more likely to result from a system designed to 
foster professional responsibility for improvement than one that relies 
on external contiol . 

By focusing on improvement, I do not believe that I sidestep 
accountability. On the contrary, if districts create systems of 
indicators that are useful to policymakers in improving education 
quality, they are necessarily useful for purposes of accountability to 
the public. The converse, howpver, is less likely to be true, as 
current uses of standardized test scores demonstrate. 



National indicators, by definition, are limited in breadth and 
depth, and, as such, are neither sensitive to differences across states 
nor detailed and specific enough to be useful to states. Local 
education agencies add two more levels of variation to the 
system- -districts and schools. Hence, the challenges facing the 
creation of a useful system of indicators are complicated even more. 

For indicators to contribute to the goal of local improvement, I 
take as given that local agencies must define their own indicators (and 
I argue that district policymakers should in turn pass on similar 
discretion to their schools). Therefore, I do not even corsider a 
multi- tiered system of indicators created above the l^vel of local 
education agencies. 

Most districts currently collect a lot of operational data; yet 
these data are rarely used to r.nform policymaking. In the next section 
I look at how indicators might be created that are more useful to 
policymakers than existing data by (1) developing measures more closely 
tied to schooling processes and (2) analyzing data in ways more 
relevant to policy issues. 

I then turn to the other side of the picture with a review of the 
literature on how policymakers incorporate data into their planning and 
decisionmaking. Although connections between data and policjrmaking are 
rare, research on organizations offers insights on how district 
characteristics can promote or inhibit attention to data. I suggest a 
set of local corditions that will facilitate uses of data for 
improvement and urge that attention be paid to developing these 
conditions as well as developing useful indicators. 
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I. CRfATING POLICY-RELEVANT INDICATORS 



To be useful for local educational improvement, indicators should 
provide adeqtiate measures of those aspects of schooling deemed 
iaportant and aseiiable to change through policymaking. A useful 
system of indicators therefore requires: (1) measures of the content 
and quality of instruction and (2) analyses and presentation of data 
directed to specific policy issues. 

CREATING INDICATORS TIED TO INSTRUCTION 

For indicators to be useful locally, they mv.st have a direct 
connection tk) the actual processes of schooling- -the content and 
quality of instruction. Because so much emphasis is placed on test 
scores » as a first step local policynakers need to be sure that the 
tests they use adequately neasure the goals of instruction. For many 
districts this may mean selecting or developing tests that are better 
matched to their curricular goals and educational philosophy than those 
currently used. 

Traditional standardized tests are limited in their ability to 
assess domains such as reading comprehension, critical thinking and 
problem solving skills. If these arr valued outcomes, new tests ought 
to be considered. Attention to this issue will result, at the least, 
in acknowledgment of the limitations of tests currently in use. It 
will also contribute to clear communication; the use of any test 
signals teachers what is expected of them (Cuban 1984; Madaus 1985, 
1986). 

Beyond test scores, the recent and growing body of research on 
effective instruction, schools, and programs suggests several fruitful 
directions for creating indicators with potential to guide improvement 
efforts . 

Educational research has made considerable progress over the last 
decade in characterizing effective instruction and the conditions under 
which it is likely to occur. For example, research connecting teacher 
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behavior to student learning demonstrates that academic learning is 
influenced by the amount of time students are engaged in appropriate 
activities, by teachers actively structuring information and relating 
it to what students already know, and by teachers* monitoring 
performance and providing feedback in a variety of modes (Brophy and 
Good, 1986). 

These findings suggest that a useful indicator would be the amount 
of time students are exposed to particular topics, which serves as the 
outside bound for the amount of time students spend in appropriate 
activities. For example, how many class periods in eighth grade math 
are devoted to graphing? If students are deemed deficient in graphing 
skills and adequate numbers of class periods are devoted to the topic, 
then the data indicate that a closer look into instructional quality is 
merited. 

Similarly, the research community has identified a number of 
school characteristics associated with effectiveness. This literature 
suggests that an effective school has a strong instructional leader, a 
climate conducive to learning and th^t it places high value on academic 
achievement, staff collegiality , high expectations for all students, 
meaningful assessment, clearly communicated rules for behavior, teacher 
involvement in decisionmaking, and parent support (Purkey and Smith 
1983). Recent studies have even demonstrated that it is possible, 
albeit time -consuming and difficult, to change an ineffective school 
into an effective one (McCormack-Larkin and Kritek, 1932; David and 
Peterson, 1984). 

Although most of these characteristics are difficult to measure 
lirectly, various direct and indirect approximations are possible. For 
example, schools might rate themselves on a checklist tha'' reflects 
whether school rules are posted in every classroom, teachers have 
planning periods and opportunities to plan together, resources 
essential to a productive climate are present and accessible (recent 
textbooks and teaching materials, photocopiers, libraries, computers, 
and supplies). District policymakers can select those characteristics 
that are consistent with their (and school staffs*) views of effective 
schools and create their own indices of correlates of effectiveness. 
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Research on effective schools finds that the culture of individual 
schools varies and is strongly associated with Instructional 
•effectiveness and commitment to improvement (Purkey and Smith 1983). 
i^ecause schools are smaller, more manageable organizational units than 
districts, these researchers conclude that a promising direction for 
education improvement lies in granting more autonomy to schools. 

Creating a set of indicators provides an opportunity for districts 
to encourage school *based i]q>rovement by Involving schools In the 
process of defining shared and unique measures. A system of indicators 
wholly controlled by the district goes in the opposite direction by 
reducing school autonomy and teacher discretion (Wise 1979). 

For example, districts might require schools to develop profiles 
structured by a set of jointly defined indicators of effectiveness, 
allowing for ^schools to add their own and comment on changes relative 
to prior years. Districts could provide schools with profiles based on 
district-wide data collection- -test score summaries, budget 
allocations, student characteristics- -and solicit comments from school 
staff, such as hypothesized explanations for changes in test scores. 
Schools might be asked to state specific goals for a three to five year 
period and document their progress on achieving the goals in such areas 
as achievement, attendance, numbers of parents involved, suspension, 
expulsion, and drop-out rates. 

In this approach, a school's past perfomance Is the frane of 
reference for Interpreting indicators. 11 attendance and test scores 
decrease while retentions and expulsions increase, district 
administrators will want more information. Spending time in the school 
is the best source of data but usually prohibitively time consuming and 
expensive. An alternative is something akin to focus groups used to 
gather information for marketing in the private sector: inviting a 
representative group of teachers and administrators to a group 
interview in which their views are solicited in a non- threatening 
environment . 
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ANALYZING EXISTING DATA TO INFORM POLICY 

Virtually all districts already administer and collect 
standardized test data at least annually. Many districts also collect 
a variety of other test data, including state or local competency 
tests, college entrance examF, and ability or aptitude tests. Yet, as 
the following example illustrates, the average test scores typically 
reported by districts do not serve policymakers well. 

The population of the junior high school in one California 
district is roughly half upper middle class whites and half low- income 
recent Mexican immigrants. The school's average test score is around 
the 50th percentile, and goes up or down a little from on year to the 
next. Policymakers have debated whether the schools' weak spot is in 
language development or adequate preparation for its college bound 
population. However, this debate occurred without an analysis of the 
data by income or ethnicity or language facility. If the scores 
reflect shifts in the proportion of advantaged and disadvantaged 
students, policymakers are debating the wrong solutions. 
Alternatively, if the data show an increase in the advantaged 
population and a corresponding decrease in test scores for that group, 
attention to the college bound program wcujd be indicated. 

District administrators generally report that test scores are not 
useful in meeting their information needs (Sproull and Zubrow, 1982). 
Yet test data reporte'l :n the aggregate serve as the primary indicator 
of effectiveness across districts. Given that the tests measure valued 
goals, the utility of test scores could be greatly enhanced by analyses 
that associate shifts in test scores with shifts in other factors. 

Districts keep records of students' grades, absenteeism, and 
suspensions or expulsions, as well as their participation in various 
activities such as sports. District files also include a considerable 
amount of background information on each student; for example, age, 
sex, ethnicity, home language, parents' occupations, par:.its' 
education, address, length of resident, and family size. Districts 
also keep track of which students are eligible for and participate in 
free or reduced lunches. 

Beyond all these student data, districts maintain data on 
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salaries, benefits, and numerous other expenditure categories 
(equipment, maintenance, transportation, security). Because federal 
and state programs often require application ar4d evaluation data, 
districts also maintain detailed information about each separately 
funded program. Districts may also maintain files of various kinds f 
survey data such as education and vocation of graduates and attitudes 
of students, teachers, parents, and administrators. These types of 
data permit analyses that go beyond summary statistics. Yet this 
wealth of information lies largely untapped. 

Although there are many reasons for the minimal use of data for 
educational improvement (discussed later), the absence of analysis and 
presentation relevant to policy issues is an obvious barrier. 
Policymakers need data that inform particular policy issues. School or 
even grade-level-within-school averages rarely do this. Such averages 
mask important differences among subpopulations which often define 
resource allocation decisions- -for example, gifted, college bound, 
language minority, disadvantaged, mentally handicapped, and so on. 
More attention to data analysis, guided by the issues facing 
policymakers, has the potential to render much existing data into 
useful indicators. Existing data can be transformed into useful 
indicators if they are analyzed and presented in ways that allow 
relevant comparisons and inferences . 

The importance of the school as an appropriate unit for 
organizational change strongly suggests that analyses should be 
performed at the school level. District inr'lcators might consist 
of graphs depicting individual schools along particular dimensions 
instead of, or in addition to, aggregate data. Observing that two 
schools are exceptionally high or low on some measure is far more 
valuable than trying to interpret a district-wide mean and standard 
deviation. Policy issues related to specific age levels or subjects 
might require analyses below the level of the school. 

To create or modify policies that will affect the outcomes, local 
policymakers need to know why changes in indicator values have 
occurred. Knowing that test scores or graduation rates have gone up or 
down is not sufficient because such facts do not suggest explanations. 
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Indicators are not intended to isolate causes but they should point to 
specific areas for further investigation. A shift in test score 
averages can be due to a change in student population, in instructional 
qualit>, in curriculum, resources, etc. The functioii of indicators is 
to narrow the area in which problems are defined and explanations 
sougiht. 

Finally, no amount of sensitive and relevant analysis will inform 
policy unless their presentation is clear and timely. The need for 
technical expertise and support for analysis and reporting is discussed 
in subsequent sections. 

DATA BURDEN AND PRIORITIES 

Because each potentially useful indicator can be applied to every 
grade level and every subject area, there are an infinite number of 
possible indicators (and an associated risk of unduly burdening staff 
and students with data collection). Without increasing the data burden 
on administrators, teacherc, and students, indicators can be created 
from existing data through more fine-tuned analyses tied to policy 
issues. To guard against data burden and interference with 
instruction, indicators must be carefully chosen to reflect local goals 
and priorities. This process needs to occur at the district level as 
well as the school level. One benefit of creating systems of 
indicators lies in this very process: directing attention to a 
discussion of goals and priorities for the district as a whole and 
individual schools faced with different sets of demands and 
constraints. 
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II. USES OF DATA BY POLICYMAKERS 



In this section, I look at the other side of the picture: how 
policymakers actually use data in their decisionmaking. It is one 
thing for indicators to be potentially useful; it is quite another to 
be actually used. 
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HOW LOCAL POLICIES ARE DEVELOPED 

Research on the uses of formal evidence in local policymaking has 
found few connections between available data and their use by 
policymakers. Reasons for the lack of connection are found in: (a) how 
policies are actually developed and (b) how individuals reach 
decisions. (A third reason derives from these two: the elusive nature 
of local policymaking and human decisionmaking makes their study 
extremely difficult.) 

School districts do not resemble models of rational 
decisionmaking; that is, one in which policymakers weigh evidence and 
make policy choices informed by the data (March and Olsen 1976; Elmore, 
1978) . Nor are districts hierarchical structures with clear lines of 
authority (Weick 1976; Meyer and Rowan, 1977). Local policies are made 
in a variety of ways, formally and informally, and by a variety of 
actors at different levels of the system. 

School district policies are formulated by school board members, 
superintendents, central office administrators (a subset of whom 
usually serve as the superintendent's cabinet), teacher orgarizations , 
and, in some cases, school staff. For particular policy decisions, 
there are likely to be various committees, drawn from the above actors 
and even including parents and other community members (Hanson 1981; 
Kennedy, 1982). Although a given district may characteristically 
follow a particular decisionmaking style, from strong central control 
to broad-based participation, there is rarely an identifiable process 
used repeatedly. Instead, both the specific individuals and the 
process itself differ from issue to issue (Knapp et al. 1986). 

Whatever cl.j local style of decisionmaking, it is difficult to 
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pinpoint when a policy Is actually made. Policies begin to take form 
In various ways. They may emerge from Informal discussions in hallways 
or over lunch. The process Is more like accretion over time than like 
a clearly Identifiable action (Lindblom and Cohen 1979; Cronbach and 
Associates, 1980; Weiss, 1980). Moreover, analytic distinctions among 
programmatic, policy, and managerial decisions are often blurred, with 
policymakers having responsibility for all three types of decisions. 

School boards typically spend their time on decisions about 
personnel, budgets, legal matters, and facilities; policies regarding 
Instructional concerns are the exception. In states with strong 
( mtrol over education, local boards must devote considerable energy to 
developing policies to Implement externally Imposed requirements. 
Similarly, It Is often a full-time job for central office 
administrators to Implement mandates from the board and the 
superintendent, leaving little time to create new policies designed to 
Improve Instruction. 

Furthermore, district policymakers are frequently pressed to make 
decisions in response to particular pressures of the moment, like 
complaints from parents, state mandates, or internal crises. One 
by-product of this reactive mode Is minimal attention given to future 
planning, or to anticipating the need for particular types of data. 
Hence data are rarely collected with specific needs in mind, which 
contributes to their lack of usefulness. 



HOW INDIVIDUALS REACH DECISIONS 

Another pr^rspective on policymakers* under-utlllzation of data 
comes from the literature on Individual decisionmaking and 
organizational policymaking. These studies rarely Identify clear 
connections between particular decisions and evidence ( Cohen and 
Caret, 1975; Nlsbett and Ross, 1980). In fact, regardless of the 
characteristics of the relevant data (Its technical accuracy or what it 
says), people tend to base decisions on their own beliefs and opinions 
(Janls and Mann, 1977). In the same vein, studies of evaluation use 
find that results from evaluations are more likely to be used when they 
are consistent with the existing beliefs of the users and not 
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contradicted by other 'nformation (David 1981; Cousins and Leitbwood, 
1986). 

Even defining use very broadly, to include influencing the climate 
of opinion, it is difficult to identify clear connections between data 
and policy decisions. Instead, local policymakers draw conclusions on 
the basis of their "working knowledge": an amalgamation of formal 
evidence, personal experiences, interests and goals, and personal 
beliefs and values (Kennecty 1982). Working knowledge serves as a 
filter for interpreting evidence and incorporates or ignores evidence 
in ways consistent with it (Lindblom and Cohen 1979; Kennedy, 1982). 
At the same time, working knowledge is not permanently fixed; it can be 
changed by the process of incorporating new evidence (Weiss 1980; 
Kennedy, 1982). 

These Vesearch findings imply that changing the kinds of data 
collected (by defining a set of indicators) is unlikely, by itself, to 
have a noticeable influence on the formation of policy in schools 
districts. On the otner hand, policymakers do not ignore data. In 
fact, they actively and continually seek out evidence to support their 
decisions; but they seek evidence unsystematically and they incorporate 
it into their working knowledge in unpredictable ways (Kennecty 1982). 

Tlie challenge is to translate the potential usefulness of 
indicators into information that polic3rmakers will incorporate into 
their ways of thinking. Indicators that are meaningful and relevant, 
as the previous section discusses, might have a greater influence than 
the research suggests since few studies have lookeu at conscious 
attempts to connect data to policy. More compelling are research 
findings connecting organizational characteristics to decisionmaking. 

Research on organizations demonstrate that the organizational 
context exercises considerable influence on how decisions are made 
(Elmore 1978; Sabatier and Mazmanian, 1979; Sproull, 1979; Cousins and 
Leithwood, 1986; McLaughlin and Pfeifer, 1986). The next section turns 
to thifi area for insights on how uses of indicators in policymaking 
might be facilitated. 
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IV. CCMDITimS THAT FROMOTE USES OF DATA 



Although systematic use of data is rare in local 
policymaking- -particularly in the domain of instructional policy--some 
districts are exceptions. These districts make a conscious effort to 
incorporate a variety of data sources into their policy deliberations. 
From conversations with policymakers and reports of others in such 
districts, in addition to the literature, I have identified five 
organizational factors that help policymakers use data to make 
decisions : 

1. a climate that supports planning and use of data; 

2. coiWitment to improvement by dis*"rict leaders; 

3. stakeholder involvement in designing the data system; 

4. technical expertise and data system support; and 

5. an action system and resources for change. 
These characteristics are elaborated below. 

CLIMATE THAT SUPPORTS PLANNING AND USE OF DATA 

For a local data system to generate information that is actually 
used, the organization must operate in a way that is prepared to 
incorporate such information. In particular, a district must be able 
to set goals, or at least determine what conditions are tolerable and 
intolerable, so that the information can be interpreted with reference 
to these goals (Cooley 1983). Without specified goals or criteria for 
acceptable conditions, the indicators themselves will become the goals 
(Johnstone 1981; Madaus, 1986). 

The appropriate climate typically occurs when both superintendent 
and board members share a belief in the importance of getting feedback 
on the system in order to plan for the future. Their views in turn 
affect how central office administrators and school staff view the 
importance and '»"ole of evaluative information. 
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DISTRICT COMMITMENT TO IMPROVEMENT 

Another condition that enables a data system to -'^e a useful 
function is a district's conBltment to improvement. xhis condition 
requires both that district leaders are committed to improvement and 
that their commitment is visible to district and school staff. 

The creation of a set of indicators, like the choice of evaluation 
instruments, is an essentially political activity. Who chooses the 
measures and for what ends, has a major influence on the quality of the 
data, its meaning, and its eventual use (Dornbusch and Scott, 1975). 
If district leaders are committed to improvement, district and school 
staff are more likely to willingly participate in and facilitate data 
coll*»ction activities. Moreover, they are more likely to implement 
changes from policies based on the data because organizational goals 
are aligned with professional ones. (McLaughlin and Pfeifer, 1986). 

To the extent that a system of Indicators is imposed on schools, 
without insuring that it is perceived as a guide to improvement, it is 
likely to have detrimental effects on the quality of the data. Staff 
will feel pressure to "look good," particularly on outcome measures, 
and will perceive the system as a threat. This situation is likely to 
result in two negative consequences. First, to increas. test scores, 
teachers will focus their instruction on what the test measures (Madaus 
1986). Second, teachers may alter testing conditions or reporting in 
ways that invalidate the data (Stringfield and Hartman 1985). 
Terminology is important. When data systems are called "accountability 
systems" (e.g., California's statewide system of school indicators), 
they are not perceived as guides co improvement. Even the term 
"monitoring" tends to carry a negative connotation, conjuring up 
visions of federal or state program auditors. 

Both the commitment of top leaders to improvement, and 
appropriately chosen terminology to communicate that commitment, 
are necessary for a system of local indicators to ::ontribute to the 
process of improvement. 

STAKEHOLDER INVOLVEMENT 

The commitment of district leaders to improvement is signalled 
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most strongly by involving stakeholder groups in the design of the 
district's data system and soliciting their views as part of the data 
collection. Data are far more likely to be credible to school and 
district staff and to the community if these groups have had an 
opportunity to suggest what should be measured and how (Patti^n 1975; 
Stake, 1975; Cuba and Lincoln, 1982). 

Districts that support use of data typically have mechanisms for 
soliciting input from teachers, principals, and parents on a regular 
basis. Whether through committees, surveys, regularly scheduled 
meetings, focus groups, or other mechanisms, these stakeholders are 
repeatedly asked their opinions on the strengths and weaknesses of the 
schools and the resources needed to change. 

Policymakers in districts with such mechanisms rely on them very 
heavily (Waliace 1986). Although they filter the information (often 
taking into account whether staff perceptions were elicited in a 
setting that facilitated frankness), these data are believed more than 
numbers (David 1981). 

TECHNICAL EXPERTISE AND SUPPORT 

An effective data system places tremendous demands on the 
capabilities of the analyst. This role requires an Individual with not 
only the requisite technical expertise but also excellent skills in 
presentation, communication, and understanding of policy and the 
organizational context in which policy is developed. The analyst needs 
to understand the information needs of the audience and communicate 
results in ways that are clear and directed to the relevant issues 
(Alkin, Daillak, and White 1979; Bickel, 1984; Cousins and Leithwood, 
1986; Shapiro, 1985). 

D^ta are more likely to be perceived as useful when there are open 
channels of communication and a common language between evaluators and 
decisionmakers (Lourie 1976) . Presenting results in person, which 
provides opportunities for clarification, and basing the presentation 
on an understanding of the context in which the audience operates, 
greatly increase the likelihood of clear communication and hence use 
(Brown and Braskamp, 1980; Raizen and Rossi, 1981), Moreover, data are 



more likely to be understood and incorporated into working knowledge 
when communication is an ongoing part of the evaluation system and not 
merely a presentation of final results (Cooley 1983; Brown and 
Braskamp, 1980; Raizen and Rossi, 1981). 

Use of outside consultants and access to technical assistance 
(provided by federal, state, local consortia, or other mechanisms) can 
strengthen districts' analytic capabilities and presentation skills. 

With the exception of very small districts, a well-designed 
computer system is an essential resource for operating a useful data 
system. An effective system Includes multiple measures for each domain 
assessed and the capability of aggregating data at the classroom, 
school, and district levels (Cooley 1983). This requires an efficient 
and flexible computer-based data collection system, including computers 
or terminals in each school, which in turn may require a significant 
Investment of resources on the district's part. Top-level commitment 
to and resources for evaluation are essential components of a 
successful evaluation system (Wise et al. 1984). 



AN ACTI3N SYSTEM AND RESOURCES FOR CHANGE 

If data are collected to provide guidance for system improvement, 
districts must have the capacity to obtain more in-depth information in 
those areas indicated deficient and to initiate action when it is 
called for. This capacity is closely connected to previously described 
conditions. The resources to obtain more detailed information can be 
considered part of the previous condition- -support for the evaluation 
unit. The capacity to initiate action is an integral part of a 
district's commitment to improvement (Fullan 1982; Cooley, 1983). 
Without the ability to initiate change, pronouncements of commitment to 
improvement cannot be taken seriously by district staff (Wise et al, 
1984) . 

The capacity to initiate action requires both resources and the 
ability to create solutions. Resources for change need not require 
additional funds. But there must be flexibility and willingness to 
significantly alter existing allocations of resources. Since the bulk 
of district budgets are tied up in staff salaries, solutions will 
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V. SUMMARY AND CONCLUSIONS 



For indicators to be useful to local policymakers in improving the 
quality of education, they must point to specific strengths and 
weaknesses as well as sources of explanations for them. Districts can 
take the following steps to accomplish this goal: 

Insure that indicators of outcomes adequately match 

curricular goals. 

Create indicators that reflect the content and quality 
of instruction using research findings as a guide. 

Coiiduct more fine-tuned analyses of existing data that 
provide insight beyond aggregate measures such as test 
score averages. 

Involve school staff in the specification, collection, 
analysis, and Interpretation of data for their site. 

Local policymakers have an infinite number of potential indicators 
to choose from. The choice should be guided by th^lr goals and 
priorities and the potential for policies to affect what is measured. 
It should also be limited to a small number of measures that do not 
interfere with instruction. 

For indicators to motivate district and school staff to change tha 
way they do things, their selection should be accompanied by attention 
to the organizational factors that promote use of feedback for 
improvement at all levels of the system. Data alone have never been 4 
catalyst for change. This suggests the need for cooperative planning 
among policymakers, analysts, and school staffs to crests s vorksbls 
system. 
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